Evaluating query log segmentation for frustration detection

We explore the problem of identifying and grouping information needs within a stream of queries. Our dataset contains sequences of queries issued by medical specialists to a domain-specific search system. Because the system allows for group logins, a sequence of queries could come from any number of specialists sharing an access point (for example, in an emergency room). We present several approaches for segmenting the stream of queries into tasks. Our ultimate goal is to detect frustrated searchers who are unsuccessful at satisfying their information needs. With that task in mind, we present an alternative evaluation measure and compare it to task-agnostic measures.